A Classification of Weakly Acyclic Games * 



Krzysztof R. Apt 1,2 and Sunil Simon 1 



1 CWI, Amsterdam, the Netherlands, 
2 ILLC, University of Amsterdam 



Abstract. Weakly acyclic games form a natural generalization of the 
class of games that have the finite improvement property (FIP). In such 
games one stipulates that from any initial joint strategy some finite im- 
provement path exists. We classify weakly acyclic games using the con- 
cept of a scheduler recently introduced in [T5]. We also show that finite 
games that can be solved by the iterated elimination of never best re- 
sponse strategies are weakly acyclic. 



1 Introduction 
1.1 Background 

Given a strategic game, when we allow the players to improve their choices on 
a unilateral basis, we are naturally brought to the concept of an improvement 
path, in which at each stage a single player who did not select a best response is 
allowed to select a better strategy. By definition every finite improvement path 
terminates in a Nash equilibrium. This suggests the finite improvement property 
(FIP), introduced in [10j . according to which every improvement path is finite. 
It is obviously a desired property of a game that in particular is satisfied by the 
congestion games, as explained in [TP] . 

However, the FIP is a very strong property and many natural games do not 
satisfy it. In particular, [9] studied the congestion games in which the payoff 
functions are players specific. These games do not have the FIP. Milchtaich 
proved that such games belong to a larger class of games (essentially introduced 
in [13]), called weakly acyclic games, in which one only stipulates that from any 
initial joint strategy some finite improvement path exists. 

Weakly acyclic games have a natural appeal because the concept of an im- 
provement path captures the idea of a possible 'interaction' resulting from play- 
ers' preference for better strategies and hence provides a link with distributed 
computing. In particular, [3] introduced a natural class of weakly acyclic games, 
which model the routing aspects on the Internet. In turn, [H] showed that for 
weakly acyclic games, a modification of the traditional no-regret algorithm yields 
almost sure convergence to a pure Nash equilibrium. Further, [5] proved that the 
existence of a unique (pure) Nash equilibrium in every subgame implies that the 
game is weakly acyclic. 



* A preliminary version of this paper appeared as [2]. 



1.2 Our work 



If we view a strategic game as a distributed system in which the players attempt 
to find a Nash equilibrium by means of a 'better response (respectively, 'best 
response) dynamics', then the property of being weakly acyclic only guarantees 
that finding a Nash equilibrium is always possible. However, such an existence 
guarantee does not help the players to find it. By adding to the game a sched- 
uler, a concept recently introduced in [T2], we ensure that the players always 
reach a Nash equilibrium, by repeatedly interacting with it. A scheduler is sim- 
ply a function that given a finite sequence of joint strategies selects a player who 
can improve his payoff in the last joint strategy. Each player interacts with the 
scheduler by submitting to it a strategy he selected. Subsequently the scheduler 
again selects a player who did not select a best response. This interaction process 
leaves open how each player selects his better (respectively, best) strategy. 

In the presence of a scheduler for a strategic game G we can view the result- 
ing interaction as a 'supergame' between the central authority represented by 
the scheduler and the players of G. The aim of the central authority is to reach a 
Nash equilibrium in spite of a limited guarantee on the behaviour of the players: 
all it can be sure of is that each selected player will select a better response 
(respectively a best response). The resulting interaction results in an improve- 
ment path (respectively a best response improvement path) . If all so generated 
improvement paths are finite, we say that the game respects the scheduler. 

By providing a classification of the schedulers we obtain a natural classifica- 
tion of weakly acyclic games. An advantage of such a classification is that given a 
weakly acyclic game we can determine under what adverse circumstances a Nash 
equilibrium still can be reached. Consequently some existing results can be im- 
proved. In particular, we show in Section how we can strengthen our recent 
result from [12] concerning a class of social network games. In turn, [3J recently 
strengthened the above mentioned theorem of [9] by showing that congestion 
games with player specific payoff functions respect every local best response 
scheduler, defined below in Section [3] 

In what follows we introduce eight natural categories of schedulers. They 
yield a classification of finite weakly acyclic games that consists of nine classes 
games that for two player games collapse into five classes. Then we study finite 
games that can be solved by the iterated elimination of never best response 
strategies (IENBR) and show that they are weakly acyclic. We also locate where 
such games fit in our classification. Finally, we define a modified notion of a 
potential using which we characterize the games that respect a scheduler. We 
also use this notion to reason about a natural class of coordination games. 

2 Preliminaries 

Assume a set N := {I, . . . , n} of players, where n > I. A strategic game for 
n players, written as (Si, . . . , S n ,pi, . . . ,p n ), consists of a non-empty set Si of 
strategies and a payoff function Pi : S\ x ■ • • x S n — > R, for each player i. 
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Fix a strategic game G :— (Si, . . . , S n ,pi, . . . ,p n )- We denote Si x ■ ■ ■ x S n 
by S, call each element s G S a joint strategy, denote the ith element of s by 
Si, and abbreviate the sequence (sj)j^ti to s_j. Occasionally we write (si,s-i) 
instead of s. Finally, we abbreviate Xj^iSj to S—i- 

We call a strategy Sj of player i a besi response to a joint strategy S-i of 
his opponents if Vs^ € Sj Pi(si, s_j) > ft(s^, s_j). If is (not) a best response 
to s_j, we say that player i selected (did not select) a best response in s. 
Next, we call a joint strategy s a (pure) Nash equilibrium if each is a best 
response to s_j, that is, if Vi G ./V Vs^ G S 1 , pi(si,s~i) > pi(s' i ,s^i). We also 
define 

BR(s) :— {i | player i selected a best response in s}, 

NBR(s) := {i | player i did not select a best response in s}. 

Further, we call a strategy of player i a better response given a joint 
strategy s if pi(s' i ,s^i) > pi(si,s^i). Following [10] a paf/i in S* is a sequence 
(s^s 2 ,...) of joint strategies such that for every k > 1 there is a player i 
such that s fe = (s^s^" 1 ) for some s^ -1 - A path is called an improve- 

ment path (respectively, a best response improvement path, in short a 
BR-improvement path) if it is maximal and for all k > 1, p%(s k ) > Pi(s k ^ 1 ) 
(respectively, sf is a best response to s 1 ^ 1 ), where i is the player who deviated 
from s . So in an improvement path each deviating player selects a better 
response, while in a BR-improvement path each deviating player selects a best 
response. 

The better response graph (respectively, the best response graph) asso- 
ciated with the game G is defined as (S, — >), where s — s> s' if (s, s') is a step in 
an improvement path (respectively, in an BR-improvement path). 

Given joint strategies s, s' G S and a player i we define 

sAs' iff S-i = s'_ i and Pi(s') > Pi(s), 

s4>s' iff s-W and s- is a best response to s'_ i . 

Recall that G has the finite improvement property (FIP), (respectively, 
the finite best response property (FBRP)) if every improvement path (re- 
spectively, every BR-improvement path) is finite. Obviously, if a game has the 
FIP or the FBRP, then it has a Nash equilibrium — it is the last clement of each 
path. Following [13 9] we call a strategic game weakly acyclic (respectively, 
BR-weakly acyclic) if for any joint strategy there exists a finite improvement 
path (respectively, BR-improvement path) that starts at it. 

In Section[H]we shall combine two n players games G '. — («S*i , • ■ • , S n , pi , . . . , Pn) 
and G' := (S[, . . . , S' n ,p' 1 , . . . ,p' n ) such that SiCiS' 1 = by means of the following 
operation: 

G a G' := (Si US' 1 ,...,S n US' n ,n,..., r n ), 

where 

!Pi(s) if s G Si X • • • X S n 
p[(s) ifs€S'i x---xS' n 
otherwise 



3 



The tbl operation is obviously associative, so it is justified to use tfcl as an k-axy 
operator on n players games. 

3 Schedulers 

In what follows we introduce some classes of weakly acyclic games. They are 
defined in terms of schedulers. By a scheduler we mean a function / that 
given finite sequence s , . . ., s k of joint strategies that does not end in a Nash 
equilibrium selects a player who did not select in s k a best response. In practice 
schedulers will be applied only to initial prefixes of improvement paths. 

Consider an improvement path p = (s , s 2 , . . .). We say that p respects a 
scheduler / if for all k smaller than the length of p we have s k+1 = (sj, a^), 
where /(s 1 , . . -,s k ) — i. We say that a strategic game respects a scheduler 
f if all improvement paths p which respect / are finite. Further, we say that 
a strategic game respects a BR-scheduler f if all BR-improvement paths p 
which respect / are finite. 

In what follows we study various types of schedulers. We say that a scheduler 
/ is state-based if for some function g : S — > R we have 

f(s\...,s k )=g(s k ). 

We say that a function g : V(N) — > N is a choice function if for all A ^ 
we have g(A) e A. Next, we say that a scheduler / is set-based if for some 
choice function g : V(N) — > N 

f(s\...,s k )=g(NBR(s k )). 

Finally, we say that a set-based scheduler / is local if the above choice function 
g satisfies the following property: 

if g(A) e BC A then g(A) = g{B). (1) 

A simple way of producing choice functions g : V(N) — > N that satisfy |T]) is 
the following. Take a permutation n of 1, . . ., n and define for A ^ 

[tt}(A) :=7r(fc), 

where k is the smallest element of N such that 7r(fc) £ A. That is, [7r](A) is the 
first element on the list 7r(l), . . ., 7r(n) that belongs to A. 

In Section [5] we shall need the following characterization result. 

Proposition 1. A choice function g : V(N) — > N satisfies (Qp iff it is of the 
form [w] for some permutation n of 1, . . ., n. 

Proof. Suppose a choice function g : V(N) — > N satisfies ([I}. Define a permuta- 
tion 7r of 1, . . ., n inductively as follows: 

tt(1) := g(N), tt(2) := g(N \ {tt(I)}), . . ., n(n) := g(N \ {vr(l), . . ., n(n - 1)}). 
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Take now a nonempty subset A of N. Let ir(k) = [7t](j4). By definition 
{tt(1), . . ., ir{k - I )} n A = and 7r(fc) £ A Let B := N \ {tt(1), . . ., 7r(fc - I)}. 
By definition g(£?) = 7r(/c). Further, AC B and 7r(fc) G ^4, so by property ([TJ) we 
have g(A) = g(B) = [tt](A). 

Next, it is straightforward to check that each function [tt] satisfies (U}. □ 

The games that respect schedulers satisfy obvious implications that we sum- 
marize in Figure [TJ Here FIP (respectively, FBRP) stands for the class of games 
that have the FIP (respectively, FBRP), WA (respectively, BRWA) for the class 
of weakly acyclic games (respectively, BR- weakly acyclic games) , Sched (respec- 
tively, Sched.BR) stands for the class of games that respect a scheduler (respec- 
tively, a BR-scheduler), etc. In what follows we clarify which implications can 
be reversed. 



FIP Local ==> Set ==> State =^> Sched > WA 




FBRP => Local br => Set BR ==> State BR => Sched BR BRWA 



Fig. 1. Dependencies between various classes of weakly acyclic games 



4 Schedulers versus state-based schedulers 

We prove here three implications which show that the classes of games Sched br, 
Sched, State and State br coincide. 

Theorem 1 {Sched => State). If a game respects a scheduler, then it respects 
a state-based scheduler. 

Proof. Fix a strategic game G = (Si, . . . , S n ,pi, . . . ,p n ). Let Y := U feGN Yfe, 
where 

— Yq := {s G S | s is a Nash equilibrium}, 

- Y k+ i := Y k U{s\3i Vs'( S As' -> s' G Y k )}. 

For each s G Y k +i\Yk, let fstate(s) i, where i is such that Vs'(sA-s' — > s' G Yk). 
If Y = S, then we can view f state as a state-based scheduler. We now prove two 
claims concerning the set Y and the scheduler f state- 
Claim 1. // a strategic game G respects a scheduler, then Y = S . 

Proof. Suppose that G respects a scheduler /. Assume by contradiction that 
Y ^ S. Take s° G S \ Y. Suppose f(s°) = ii. By the definition of Y there exists 

a joint strategy s 1 such that s°-Vs 1 and s 1 G S\Y. Suppose /((s^s 1 )) = %2- 

Again, by the definition of Y there exists a joint strategy s 2 such that s 1 ^-s 2 
and s 2 G S \ Y. Iterating this argument we construct an infinite improvement 
path which respects /, which yields a contradiction. □ 
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Claim 2. If for a strategic game G we have Y = S, then G respects j "state- 
Proof. We prove by induction on k that all improvements paths that start in a 
joint strategy from Yp. and respect f state are finite. 

The claim holds vacuously for k = 0. Suppose it holds for some k > 0. Take 
some s £ Yfc+i \ Yfc and an improvement path £ that respects f state and starts in 
s. Suppose that fstate(s) ■= i- Then for some s' , sAs' is the first step in £. By 
the definition of f state, s' £ Yfc, so by the induction hypothesis £ is finite. □ 

Suppose now that a game G respects a scheduler. By Claim [1] Y = S, so 
f state is a state-based scheduler. By Claim [2] G respects f state- □ 

The above proof uses a construction similar to the one used to compute the 
winning regions of reachability games, see, e.g., [5J page 104]. 

Theorem 2 (Sched.BR => StatesR). If a game respects a BR- scheduler, then 
it respects a state-based BR-scheduler. 

Proof. The proof is the same as that of Theorem Q] with the relation =4> used 
instead of A. □ 

Theorem 3 (SchedsR =>■ Sched). If a finite game respects a BR-scheduler, 
then it respects a scheduler. 

Proof. The idea of the proof is as follows. Suppose that a game respects a 
BR-scheduler fsR- We construct then a scheduler / inductively by repeatedly 
scheduling the same player until he plays a best response, and subsequently 
scheduling the same player as fsR does. 

To make it precise we need some notation. We call an initial prefix of an im- 
provement path an improvement sequence. To indicate the deviating players 
at each step of an improvement sequence (s°, . . ., s k ) we shall write it alterna- 
tively as 

Given an improvement sequence £ we denote by [£]rr the subsequence of it 
obtained by deleting the joint strategies that do not result from a selection of a 
best response. In general [£;]br. is not a improvement sequence (for example, it 
does not need to be a maximal sequence), but it is if every maximal subsequence 
of it of the form s° As 1 A. . .As" 1 ends with a selection of a best response. 

Given a finite sequence of joint strategies £ we denote its last element by 
last({;) and denote the extension of £ by a joint strategy s by £, s. We define the 
desired scheduler / inductively by the length of the sequences. For a sequence 
of length 1, so a joint strategy that is not a Nash equilibrium, we put 

/(*) := fB R (s). 
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Suppose now that we denned / on all sequences of length k. Consider a 

/(£) 

sequence £, s of length k + 1. If £, s is not an improvement path or last(£) — I s 
does not hold, then we define /(£, s) arbitrarily. Otherwise we put 

/(£ s ) ' _ j*^^' if Si is a beSt res P° nse to s ~i 

]/(£) otherwise 

We claim that G respects the scheduler /. To see it take an improvement 
path £ that respects /. By the definition of /, [£]br is an improvement sequence 
that respects Jbr- By assumption [£\br is finite, so £ is finite, as well. □ 

Note that the above theorem fails to hold for infinite games. Indeed, consider 
a two player game ({0}, [0, l],pi,p2), where [0,1] stands for the real interval 
{r | < r < 1} and Pi(0, S2) = P2(0, s 2 ) := s 2 . Then 1 is a unique best response 
of player 2 to the strategy 0, so this game respects the unique BR-schcduler. 
However, it does not respect the unique scheduler. 



5 Two player games 

For two player games more implications hold. 

Proposition 2 (Sched =$> FBRP). If a two player game respects a scheduler, 
then every BR-improvement path is finite. 

Proof. Suppose that a two player game G respects a scheduler. Note that the 
best response graph of G has the property that for every node s that is not a 
source node, the set NBR(s) has at most one element. Take a BR-improvement 
path £. Suppose that (s, s') is the first step in £ and that r\ is the suffix of £ 
that starts at s'. Then every element s" of r\ is such that the set NBR(s") has 
at most one element. Hence 77 respects any scheduler and consequently is finite. 
So £ is finite, as well. □ 

We also have the following examples showing that other implications do not 
hold. 

Example 1 (Local ^> FIP). Consider the following game 



L C R 



1,0 


0,1 


0,2 


0,1 


1,0 


0,0 



It respects the local scheduler / for which /({l, 2}) = 2. However, this game 
does not have the FIP. □ 

Example 2 (State ^> Set, FBRP ^ Set). Consider the game 
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B C 



A 
B 

C 



2,2 


2,0 


0,1 


0,2 


1,1 


1,0 


1,0 


0,1 


0,0 



In Figure[2]we display the better response graph and the best response graphs 
associated with it. 




(A,C) 



(B,C) 



(C,C) 



(A,C) 



(B,C) 




(C,C) 



Fig. 2. Better response and best response graphs 



First we show that this game respects a state-based scheduler. To define it 
suffices to consider the joint strategies in which none of the players selected a 
best response. These are (A, C), (C,A) and (B,B). We put 

f(A C) :=2, f(C,A) :=1, f(B, B) := 1. 

(/(£>, B) := 2 would do, as well.) Using Figure [Ha) it is easy to check that any 
improvement path that respects this scheduler ends in (A, A). Further, the graph 
given in Figure [21(b) is acyclic, that is, this game has the FBRP. 

However, this game does not respect any set-based scheduler. Indeed, sup- 
pose otherwise. Then such a scheduler is defined using a choice function g. If 
g({l, 2}) = 1, then the infinite improvement path ({B,B), (A, B), (A,C), {B,C))* 
respects this scheduler. In turn, if g({l,2}) = 2, then the infinite improvement 
path {{B,B), (B, A), (C,A), (C,B))* respects this scheduler. □ 

Example 3 (BRWA ^> SchedsR). Consider the following game 





L 


C 


R 


T 


0,1 


1,0 


0,1 


B 


1,0 


0,1 


0,0 



It is BR-weakly acyclic. However, this game respects no BR-schedulcr. In- 
deed, there is only one scheduler for this game and the following infinite im- 
provement path respects it: 

((T,L), (B, L), (B,C), (T,C))*. 



□ 



S 



Example 4 ( WA ^> BRWA). Consider now the following game 



L C R 



0,2 


1,0 


0,1 


1,0 


0,1 


0,0 


0,0 


0,0 


1,0 



It is weakly acyclic. However, it is not BR-weakly acyclic, because the infi- 
nite BR-improvement path ((T, L), (M, L), (M,C), (T,C))* is a unique BR- 
improvement path starting at (T,L). □ 

By definition if a two player game respects a set-based scheduler, then it 
respects a local scheduler. So putting together the results obtained so far we 
get the implications and equivalences depicted in Figure All implications are 
proper. 



FIP : 



Local, Set 



FBRP, LocalBR, Setsu, 
State, StateBR, 
Sched, Sched.BR 



BRWA-. 



WA 



Fig. 3. Classification of two player finite weakly acyclic games 



As an illustration consider a two player congestion game with player-specific 
payoff functions (actually a class of games) analyzed in [S] page 115]. Each player 
has three strategies numbered 1, 2, and 3. We omit the description of the game 
and mention only its relevant characteristics. The game has two Nash equilibria, 
(1,2) and (2,1), and an infinite improvement path ((3, 2), (2,2), (2,3), (1,3), (1,1), (3,1))*. 
Additionally Pl {l, 2) > 1), 2) > Pl (2, 2), and p 2 (2, 1) > p 2 {2, 3). 

This implies that the graph depicted in Figure [4] is a subgraph of the better 
response graph of the game. The dotted edges are implied by the fact that 
(1,2) and (2,1) are Nash equilibria, the continuous edges correspond to the 
infinite improvement path, while the dashed edges are implied by the mentioned 
properties of the payoff functions. The relation between the nodes (2,1) and 
(3, 1) and the status of edges involving (3, 3) are unspecified, though the edge 
(2, 1) — > (3, 1) is excluded since (2, 1) is a Nash equilibrium. 

Note that this game respects a set-based scheduler g such that g{{l, 2}) = 2. 
Indeed, this choice allows us to 'exit' the infinite improvement path both at (1, 1), 
(2, 2) and (2, 3). One can also check that this choice properly takes care of any 
legal completion of the graph depicted in Figure 0] to a possible better response 
graph. For instance, addition of the edges (3, 2) — > (3, 3) and (3, 3) — > (2, 3) 
would create another infinite improvement path that would be 'exited' at (1, 1). 
We conclude that each game with the above characteristics belongs to Set\ FIP. 
Figure E2 implies that each such game also belongs to FBRP. 
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(3,1) 



(3,2) 



(3,3) 



Fig. 4. Subgraph of a better response graph 



6 Remaining implications 



To deal with the remaining implications we need games with at least three 
players. 

Example 5 (LocoIbr ^> FBRP). We consider a three player game in which 
every player has two strategies, and 1, and such that its best response graph 
is as shown in Figure 



(0,1,1) 



(i,i,i) 



(0,0,1) 



(1,0,1) 



(0,1,0) 



/ 

(0,0,0) 



(1,1,0) 



(1,0,0) 



Fig. 5. A best response graph 



To define the corresponding payoff functions we just need to interpret each 
directed edge (a, b, c) — !> (a, 6', c) as the statement P2 (a, b, c) = and p 2 (a, b'.c) = 

1, each dotted edge (a, b, c) (a,b',c) as the statement j?2(«, b, c) = and 

P2(a,b',c) — 0, and similarly for the other edges. This yields 

pi(0,l,l)=pi(l,0,l)=j> a (0 ) 0,l)=pa(l ) l,l)=P3(l,0 ) 0) := 1, 

with the remaining payoffs equal to 0. 

This game respects any local BR-scheduler / for which /({2, 3}) = 3. How- 
ever, this game does not have the FBRP. □ 

Example 6 (Set ^> Local, SetBR ^> LocoIbr)- We first construct four three 
player games. Every player in each game has two strategies, and 1. The better 
response graphs of these games are depicted in Figure [5J To define the corre- 
sponding payoff functions we proceed as in Example 
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(0,1,1) 



(1,1,1) 



(0,1,1) 



(1,1,1) 



(0,0,1) 



(1,0,1) 



(0,0,1) 



(1,0,1) 



(0,1,0) 



(1,1,0) 



(0,1,0) 



(1,1,0) 



(0,0,0) 



(1,0,0) 



(0,0,0) 



(1,0,0) 



(0,1,1) (1,1,1) (0,1,1) (1,1,1) 



(0,0,1) -< (1,0,1) (0,0,1) ' 



(0,1,0) 



■ (1,1,0) 

/ 



(1,0,1) 



(0,1,0) 



(1,1,0) 



(0,0,0) 9- (1,0,0) (0,0,0) 



(1,0,0) 



Fig. 6. Four better response graphs 



Next, we make for each player his strategy sets in these four games mutually 
disjoint by renaming the strategies and 1 in the zth game to Oj and lj. Denote 
the resulting games by G\, . . ., G4. Let G := G\ W G2 W G3 U G4. In the better 
response graph of G there are precisely three loops that correspond to the ones 
depicted in Figure^ Using this figure one can check that G respects the following 
set-based scheduler: 

/({2,3}) :=3, /({1,3}) :=1, /({1,2}) := 2, 

with an arbitrary value for the input {1, 2, 3}. 

However, G does not respect any local scheduler, since to 'exit' each of these 
three loops each set-based scheduler needs to make the above selections, and 
then no selection for the input {1, 2, 3} can make the scheduler local. 

The above reasoning also holds for the BR-schedulers, since every better 
response in G is also a best response, as all payoffs in G are either or 1. □ 

Example 7 (StatesR ^> SetsR)- We first construct two three player games. 
Every player in each game has two strategies, and 1. The best response graphs 
of these games are depicted in Figure H The first one coincides with the first 
better response graph of Figure El 

Denote these games by Gi and G2 and let G := G% W G2. Then G respects 
the following state-based BR-scheduler (we use here the same renaming of the 
strategies as in Example E]): 

/(lx.Qi.li) :=3, /(1 2 ,0 2 ,1 2 ) :=2. 
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(0,1,1) 



(1,1,1) 



(0,1,1) 



(1,1,1) 



(0,0,1) 

I 

! 



(1,0,1) 



I (0,1,0) 



I / 

(0,0,0) 



(1,1,0) 



(1,0,0) 



(0,0,1) 
A 



(0,0,0) 



(1,0,1) 



(0,1,0) 



(1,1,0) 



(1,0,0) 



Fig. 7. Two best response graphs 



However, G does not respect any set-based BR-scheduler, since in each of the 
above two joint strategies the set of players who did not select a best response is 
the same (namely {2, 3}), and to 'exit' each of the two loops in the corresponding 
best response graphs one needs to take the above choices. □ 

This brings us to the following final diagram depicted in Figure [8] All impli- 
cations are proper. 



FBRP 



FIP 



Local i 



Local 



Setr 



State, State br, 
Sched , Sched br 



BRWA 



WA 



Set 



Fig. 8. Classification of finite weakly acyclic games 



Let us mention here that the above classification can be extended by also 
considering for each class S of schedulers the finite games that respect every (as 
opposed to some) scheduler in the class S. This was done by Ruben Brokkelkamp 
and Mees de Vries (private communication). 



7 Games that can be solved by the iterated elimination 
of NBR 

Weakly acyclic games are of natural interest because they have (pure) Nash 
equilibria. In the literature another class of finite games has been studied with 
the same property, namely those that can be solved by the iterated elimination 
of strictly dominated strategies. In what follows we focus on a more general 
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class of finite games that can be solved by the iterated elimination of never best 
responses (IENBR), where we limit ourselves to never best responses to joint 
pure strategies of the opponents. 

Given a strategic game G = (Si, . . . ,S n ,pi, . . . ,p n ) we say that a strategy 
Si G Si is a never best response in G if for every s_; G S-i there is a strategy 
s- G Si such that p^, S_j) > P;(s;, S_j). 

To better understand the concept of a never best response recall that given 
a finite strategic game G = (Si, . . . , S n ,pi, . . . ,p n ) a mixed strategy for player 
i is a probability distribution over Si. Further, recall that a strategy Si G 5 1 , is 
strictly dominated by a mixed strategy if there exists a mixed strategy rrii 
such that for all s_i G S-i, pi(m,i, s-i) > pi(s i} s-i), where Pi(mi, s-i) is defined 
in the expected way. 

Then it is straightforward to check that if Si is strictly dominated by a mixed 
strategy then Si is a never best response. Also, the reverse implication does not 
hold. 

Given a binary relation — > we denote by — >* its transitive reflexive closure. 
In what follows we consider a specific relation — > between games defined as 
follows: 

G^G' 

where G = (Si, . . . , S n ,pi, . . . ,p n ), G' = (S[, . . . , S' n ,p[, . . . ,p' n ) is a proper 
subgame of G, and 

Vi G {1, . . . , n} Vsi G Si \ SI ~<3s-i G S-i Si is a best response to s-i in S. 

That is, G — > G' when G' results from G by removing from it some strategies 
that are never best responses. 

If for some game G" in which each player is left with exactly one strategy we 
have G ^* G", then we say that G can be solved by the iterated elimination of 
never best responses, in short can be solved by IENBR. 

The games that can be solved by IENBR are of interest because of the fol- 
lowing observation. 

Proposition 3. // a finite game can be solved by IENBR then it has a unique 
Nash equilibrium. 

Proof. It suffices to note that each step of the elimination process maintains the 
set of Nash equilibria. □ 

We now prove that this class of games is actually contained in the class of 
weakly acyclic games. More precisely, the following holds. 

Theorem 4. // a finite game can be solved by IENBR, then it is in StatesR- 

Proof. We use the scheduler / which schedules each time the player who did not 
play a best response the longest, breaking ties in favour of the player with the 
smallest index. Fix a finite game G that can be solved by IENBR. By definition 
of the scheduler /, it satisfies the following property: 
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(PI) for all i £ N, for every BR-improvement path p = s°, s 1 , . . . in G that 
respects / and for all j > 0, there exists k > j such that i £ BR(s k ). 

Let p = s , s , . . . a BR-improvement path which respects /. We argue that 
p is finite. 

We find it useful to introduce the following notion: a finite sequence of strat- 
egy profiles b — s°, . . . , s k is called a block if for all i £ NBR(s Q ), there exists 
j £ {0, . . . , k} such that either i £ BR(s J ) or i £ f(s°, . . . , s J ). The sequence b 
is said to be minimal if there is no prefix b' of b such that b 1 is a block. Since 
/ satisfies (PI), we can represent the best response improvement path p as a 
concatenation of minimal blocks b°, b\ . . .. 

Since G can be solved by IENBR, there is a finite reduction sequence £ = 
G° — > G 1 — > . . . — > G m for G such that each player is left with a single strategy 
in G m . By Theorem 4.2 in pQ we can assume without loss of generality that each 
reduction step eliminates a single strategy. Let s^), (12, Sj 2 ), . . . , (i m , s^ m ) be 
the corresponding sequence of pairs of player identity and of the strategy which 
are eliminated in £. Consider the first pair (ii, s^) and the block b° = s°, . . . , s . 
We have two cases: 

- s° =s- 

Then ii £ NBR(s°). Indeed, since Sjj is a never best response, i\ £ NBR(s) 
for any s £ S. Since / satisfies (PI) and is a never best response there 
exists j £ {0, . . . , 1} such that i\ £ f(s°, . . . , s J ). Therefore s^ +1 ^ . Again, 
since is a never best response player i never switches back to Si 1 in the 
suffix of p starting in s- 7 irrespective of the strategies chosen by the other 
players. 

Then since is a never best response, player i\ never switches to in 
the improvement path p, irrespective of the strategies chosen by the other 
players. 

Therefore in the suffix p 1 = b , b 2 , 6 3 , . . . of the improvement path p the strategy 
Sij is never chosen. This means that p 1 is a best response improvement path in 
the restricted game G 1 . Now consider the pair Sj 2 ) and the block b 1 . By the 
same argument as above, we can show that in the suffix p 2 = b 2 ,b 3 , . . . of the 
improvement path p, the strategy Si 2 is never chosen and therefore p 2 is a best 
response improvement path in the restricted game G 2 . Since G can be solved by 
the elimination sequence £ = G°, G 1 , . . . , G m , by iterating the above argument 
we get that there is a suffix of the improvement path p which involves only joint 
strategies in G rn . Since in G m each player is left with a single strategy, p is finite. 

We conclude that the game G respects the BR-scheduler /. Consequently, by 
Theorem [21 G is in State br- □ 

In particular, in view of the remarks at the beginning of this section, if a 
game can be solved by iterated elimination of strictly dominated strategies (by 
a mixed strategy), in short, IESDS, then it is weakly acyclic. An example of such 
a game is the beauty contest game due to [TT]. In this game there are n > 2 
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players, each with the set of strategies equal {1, . . . , 100}. Each player submits a 
number and the payoff to each player is obtained by splitting 1 equally between 
the players whose submitted number is closest to | of the average. For example, 
if the submissions are 29,32,29, then the payoffs are respectively i,0, i. By 
Theorem 2] this game is weakly acyclic. 

Note that the implication in the above theorem cannot be reversed. Indeed, 
the game given in Example [7] is in States r and it is easy to check that it cannot 
be solved by IENBR, as every strategy in this game is a best response. An even 
simpler example is the well-known Battle of Sexes game. 

We now identify the smallest class used in our classification of finite weakly 
acyclic games that includes all finite games that can be solved by IENBR. We 
begin with two player games for which we have the classification presented in 
Figure [31 of Section [5] In view of this, it suffices to note the following example. 

Example 8 (IENBR ^> Set). Consider the game 

ABC 



A 


3. 


2 


1- 


1 


2 





B 




2 


3. 


1 


1 


2 


C 


2- 


2 


2. 


3 


1 


1 



and its better response graph displayed in Figure [5] 



(3,2) ^ (1,1) ^ (2,0) 




(2,2) s- (2,3) ^ (1,1) 




Fig. 9. A better response graph 



It is easy to see that this game can be solved by IENBR and that the outcome 
is (A, A). However, this game does not respect any set-based scheduler. Indeed, 
suppose otherwise. Then such a scheduler is defined using a choice function g. If 
g({l, 2}) = 1, then the infinite improvement path ((A, B), (B, B), (B,A), (A, A))* 
respects this scheduler. In turn, if g({l,2}) = 2, then the infinite improvement 
path ((C, A), (C,B), (B, B), (B,A))* respects this scheduler. □ 

The case of arbitrary games is more complicated. First we show how given two 
n player games Gi := (Si, . . .,S n ,pi, . . . ,p n ) and G 2 ■= (S[, . . .,S' n ,n, . . . ,r„) 
that have strictly positive payoffs and both can be solved by IENBR, we can 
construct using an auxiliary game G3 a game G := Gi W G2 W G3 that also can 
be solved by IENBR. 
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Suppose that the outcomes of the IENBR applied to the games G\ and G 2 
are respectively the joint strategies (si, . . ., s n ) and (s[, . . ., s' n ). Further, assume 
that a^SiU S[. Let G3 := ({a}, 5*2 U S' 2 , . . . , S'n U S' n , u\, . . . , u n ), where the Ui 
payoff functions are defined in such a way that 

— the NBR eliminations steps applied to G\ remain valid in the context of G, 

— the same holds for the game G2, 

— the resulting game ({a, s\, s[}, {S2, s' 2 }, . . . , {s n , s' n }, u%, . . . , u n ) can be solved 
by IENBR. 

The first two items can be achieved by ensuring that for each i G {2, . . ., n} 
and each eliminated strategy s" G (Si U 5,-) \ {s,;, s-} is not a best response to a 
joint strategy S-i in G with si = a. We thus stipulate that 

Ui(a, s_i) := 0, when € (S^ U S 1 ^) \ {s^, s^} for some i G {2, . . ., n}. 

To solve the game ({o, s±, s[}, {S2, s 2 }, . . . , {s n , s' n }, u\, . . . , u n ) by IENBR 
we first stipulate that the strategies s± and are strictly dominated in it by a. 
We thus stipulate that 



where max is the maximum payoff used in G\ or G2 . 

To ensure that the game ({a}, {S2, s' 2 }, ■ ■ ■ , {s n , s' n }, Ui, . . . , u n ) can be solved 
by IENBR we further stipulate that for i G {2, . . ., n} 



Then for any payoff functions u%,.. .,u n that satisfy the above conditions 
the outcome of the IENBR applied to above game G yields the joint strategy 
(a, s 2) . . -,s n ). 

We now proceed with the following example. 

Example 9 (IENBR i> Local BR ). 
First, consider Figure [TU1 



Ui(a, s_i) := max+1, when s_i G x jjti{sj, s'j}, 



M 4 (si,s_i) > w 4 (s-,s_i), where s_ 4 G {a} x^^i {sj, s^}. 



(0,1,1) 



(1,1,1) 




(0,1,0) - - 



- - (1,1,0) 





(0,0,0) 



(1,0,0) 



Fig. 10. A best response graph 



1G 



It depicts the best response graph of a three player game G\ . We define the 
payoff functions analogously to Example Note that this game can be solved 
by IENBR with the outcome being the joint strategy (0, 0, 0). 

Further, this game respects a set-based BR-scheduler / iff 

/({l,3}) = 3or/({2,3}) = 3. 

So it does not respect the local BR-schedulers that correspond to the permuta- 
tions 1, 2, 3 and 2, 1, 3. 

By renaming the players we obtain two other games, G2 and G3, that both 
can be solved by IENBR and such that G2 does not respect the local BR- 
schedulers that correspond to the permutations 1, 3, 2 and 3, 1, 2 and G3 does not 
respect the local BR-schedulers that correspond to the permutations 3, 2, 1 and 
2, 3, 1. Using the construction explained prior to this example twice we construct 
then a game G that can be solved by IENBR and that does not respect any local 
BR-scheduler. □ 

So this example shows that the class of finite games that can be solved by 
IENBR is not contained in LocclIbr- We suspect that in fact this class is not 
contained in ScAbr, however were not able to prove it. 



8 Potentials 

To characterize the finite games that have the FIP [TU] introduced the concept of 
a (generalized ordinal) potential. We now introduce an appropriately modified 
notion to characterize the games that respect a scheduler. We shall use it in the 
next section to reason about a natural class of games. 

Consider a game (Si, . . . , S n ,pi, . . . ,p n ) and a scheduler /. We say that a 
function F : S — > R is an / -potential (respectively, an f -BR-potential) if for 
every initial prefix of an improvement path (respectively, an BR-improvemcnt 
path) (a 1 , . . ., s k , s k+1 ) that respects / we have F(s k+1 ) > F(s k ). 

Note that when / is a state-based scheduler, then a function F is an f- 
potential iff for all i, s[ and s 

if f(s) = i and pi(s^, s_ 4 ) > p,(sj, s_<), then F(s' i , s_;) > F(s l , s_ 4 ), 

and similarly for the /-BR-potentials. In the proof below we use the following 
classic result of [7]. 

Lemma 1 (Konig's Lemma). Any finitely branching tree is either finite or it 
has an infinite path. 

Theorem 5. Consider a finite game G. 

(i) G respects a scheduler f iff an f -potential exists, 
(ii) G respects a BR-scheduler f iff an f -BR-potential exists. 
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Proof, (i) ( => ) Consider a branching tree the root of which has all joint strate- 
gies as successors, the non-root elements of which are joint strategies, and whose 
branches are the improvement paths that respect /. Because the game is finite 
this tree is finitely branching. By Konig's Lemma this tree is finite, so we con- 
clude that the number of improvement paths that respect / is finite. Given a 
joint strategy s define F(s) to be the number of improvement sequences (in the 
sense of the proof of Theorem [3]) that respect / and that terminate in s. Clearly 
F is an /-potential. 

( <= ) Let F be an /-potential. Suppose by contradiction that an infinite im- 
provement path that respects / exists. Then the corresponding values of F form 
a strictly increasing infinite sequence. This is a contradiction, since there are 
only finitely many joint strategies. 

The proof of (ii) is analogous. □ 

The argument given in (i)( ) follows the proof of [5] of the fact that every 
game that has the F1P has a generalized ordinal potential. Note that when the 
range of the /-potential is finite the implications (<=) in (i) and (ii) also hold 
for infinite games. 

9 Cyclic coordination games 

In coordination games the players need to coordinate their strategies in order 
to choose among multiple pure Nash equilibria. Here we consider a natural set 
up according to which the players are arranged in a directed simple cycle and 
the payoff functions can yield three values: if one chooses a 'noncommitting' 
strategy, 1 if one coordinates with the neighbour and —1 otherwise. We call 
such games cyclic coordination games. They are special cases of strategic 
games introduced in [12j that are naturally associated with social networks built 
over arbitrary weighted directed graphs. We showed there that a similar game 
respects a local scheduler. We now show how using the concept of an /-potential 
we can strengthen this result. 

More precisely, let G coor d — (S%, . . . , S n ,pi, . . . ,p n ) be a (possibly infinite) 
strategic game in which there is a special strategy to G DieiV &i common to all 
the players. For i G N, let i © 1 and i © 1 denote the increment and decrement 
operations done in cyclic order within {1, . . . , n}. That is, for i G {1, . . ., n — 1}, 
i © 1 = i + 1, n © 1 = 1, for i G {2, . . ., n}, i © 1 = i - 1, and 1 © 1 = n. The 
payoff functions are defined as, 



Theorem 6. Each coordination game G coor d respects every local scheduler. 

Proof. For n = 2, it is easy to see that G coor d has the FIP and hence the result 
follows. Therefore, assume that n > 2. We prove the result by showing that for 
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every local scheduler /, it is possible to associate an /-potential with the game 

G coord- 

Let / be a local scheduler. By Proposition!]] the choice function g associated 
with / is of the form [w] for some permutation it of 1,.. .,n. Let I = n(n) be 
the last element in the permutation tt (this will be the only information about 7r 
that we shall rely upon). Let U := { — 1,0, 1}™ and let F : S — > U be a function 
defined by F(s) := (pi(s),p ;e i(s),p ie2 (s), . . . , j>j©(„_i)(s)). 

For x 6 U and i G {1, . . . , n}, Xi denotes the i-th entry in x and as before, 
X—i = (x2, ■ • ■ , x n ). We also use the notation F(s)[i] to denote the i-th entry in 
the n-tuple F(s). 

Let -<l be the strict counterpart of the lexicographic ordering over the (n— 1)- 
tuplcs of —1,0, 1, where -1 -<i -<i 1. We extend ^ttoa relation <a C U x U . 
For x,y € U such that x ^ y, x <\ y if one of the following mutually exclusive 
conditions holds: 

CI xi G {-1, 1} and yi = 0, 

C2 xi = yi = and X-i < L y~i, 

C3 xi,yi G {-1, 1} and X-i < L y_i, 

C4 xi,y\ e {-1, 1}, x-i = y-i and x\ -< L yi. 

In other words, if the first entry of y is and that of x is not 0, then x <\y. If 
the first entry of both x and y is 0, then to order x and y we use the lexicographic 
ordering over the (n — l)-tuples x^i and y-\. If the first entry of both x and y 
is not 0, then again to order x and y we use the lexicographic ordering over X-\ 
and the exception being when x_i = In this case, to determine the 
ordering we use the lexicographic ordering over x\ and y±. 

Claim 3. The relation < is a strict total ordering over U . 

Assuming Claim [31 consider an initial prefix ^k+i — (s 1 , ■ • • , s k , s k+1 ) of an 
improvement path £ that respects /. We claim that F(s k ) < F(s k+1 ). We have 
the following cases: 

~ f(s k ) — l(Bi where i G {1, . . . , n— 1}. Since £ respects /, we have pi@i{s k ) < 
Pm{s k+1 ), so F{s k )[i + 1] -<£ F(s' £ + 1 )[i + 1]. Since i ^ n, if i > 1, then by 
the definition of the payoff functions, for all j G {1, . . . , i — 1}, Pi(Bj{s k ) = 
piQj(s k+1 ). If i ^ n — 1, then pi(s k ) = pi(s k+1 ) and it immediately follows 
that F(s k ) <l i ;l (s fe+1 ). Therefore, the interesting case is when i = n— 1. Here 
we show that the first entry in F(s k+1 ) remains after the update by player 
n — 1 iff the first entry in F(s k ) is 0. 

• If F(s k )[l] = 0, then F(s fc+1 )[l] = 0. 

Indeed, suppose F(s k )[l] = 0. Since f(s k ) ^ I, we have sf = s k+1 . By 
the definition of the payoff function, for any joint strategy s, pi(s) = iff 
si = t . Thus irrespective of the choice of l(B(n — 1) we have pi{s k+1 ) = 0, 
so F(s fe+1 )[l] = 0. 
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• If F(s k )[l] ^ 0, then F(s k+1 )[l] ^ 0. 

Suppose F(s fe )[l] 7^ 0. By the definition of the payoff functions, s k ^ to. 
Since f(s k ) ^ I, we have sf = s k+1 . Therefore irrespective of the choice 
of I © (n - 1) we have pii-s^ 1 ) ^ 0, so F{s k+1 )[l] ^ 0. 
Thus by conditions C2 and C3 in the definition of <d, and the fact that 
(F(s fe ))_i < L (F(s' £ + 1 ))_i, we indeed have F{s k ) < F(s k+1 ). 

~ f( sk ) = Since £ respects /, for all i £ {1, . . . ,n— 1} we have Zffii £ BR(s k ). 
We claim that in this case, sf ^ t an d s fei — *o- Suppose not. If sf = t , 
then for alH £ {1, . . . , n — 1}, I © i £ BR(s k ) implies that sf ei = to- This in 
turn implies that I £ BR(s k ), which is a contradiction. If sf el 7^ to, then we 
have the following two possibilities: 

• sf el = sf . This implies / £ BR(s k ), which is a contradiction. 

• s fei 7^ s i- Then there exists j £ {1, .. . ,n — 1} such that s* ffl - — sf el 
and sf ffi (j_i) 7^ sfej- Since sf ej - 7^ to, this implies that Zffi j ^ BR(s k ), 
which is a contradiction. 

Now, if sf 7^ t , sf el = t and pi(s k ) < pi(s k+1 ), then s k+1 = t . By CI in 
the definition of <, it then follows that F(s k ) < F(s k+1 ). 

Finally, since the set U which is the range of the function F is finite and 
<3 is a strict total order on U, we can use an appropriate encoding e : U —> M. 
such that Mi < u 2 iff e(u\) < e(u 2 ). Then e(F(s k )) < e(F(s k+1 )). So e o F is an 
/-potential. By the remark following Theorem [5] the result follows. □ 

Proof of Claim[3[ Let x,y £ U such that x 7^ y. We have the following cases. 

— xx £ {-1, 1} and yi = 0. Then by CI, x < y. 

— x\ = and y\ = 0. Then by C2, if x~\ <l V-i then x <jy else y < x. 

— x\ — and 2/1 £ { — 1, !}• Then by CI, y < x. 

~ £1,2/1 £ { — 1) 1} an( l X-i 7^ Then by C3, if X-\ <l y~i then x < y else 
y < x. 

~ ^1)2/1 £ { — 1, 1]- and x_i = Then by C4, if xi -<£ 1/1 then x < y else 
y <} x. 

Further, it can be verified that the relation < is transitive by a straightforward 
case analysis. □ 

Note that Theorem E] cannot be extended to set-based schedulers. Indeed, 
suppose that n > 2, and for some t 7^ to we have t £ Di^Si. Consider the 
joint strategy s := (t,t , . . .,t ) and a set-based scheduler / such that for all 
k £ {1, . . ., n}, /({&, Ac © 1}) := k © 1, f({k, k © 2}) := k, with arbitrary values 
for other inputs. Then the following infinite improvement path respects this 
scheduler. For the sake of readability we underlined the strategies that are not 
best responses. 

(t, to, . . ., to), (t, t, to, . . ., to), (to, t, to, . . ., to), . . . 
Finally, observe the following properties of the coordination games. 
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Theorem 7. 



(i) The game G coor d has the FIP iff n = 2 or D^nSi = {to}, 
(ii) In G coor d, starting from each joint strategy there exists an improvement 
path of length < n and a BR- improvement path of length < 2n — 2. 

Note that s is a Nash equilibrium in the game G COO rd iff it is is of the form 
(t, . . . , t). So we can alternatively state item (i) as: The game G coor d has the FIP 
iff n = 2 or it has exactly one Nash equilibrium. 

Proof, (i) (=>) As already mentioned when n = 2, G COO rd has the FIP. If n > 2, 
then the above example implies that Rig at Si = {to}- 

(<=) Suppose that G COO rd docs not have the FIP. Consider an infinite improve- 
ment path £. Some player, say i, is selected in £ infinitely often. This means that 
player i selects in £ some strategy t ^ to infinitely often. Indeed, otherwise from 
some moment on in each joint strategy in £ his strategy would be to, which is 
not the case. 

Each time player i switches in £ to the strategy t, the strategy of his pre- 
decessor i Q 1 is necessarily t, as well. So also player i Q 1 switches in £ to t 
infinitely often. Iterating this reasoning we conclude that each player selects in 
£ the strategy t infinitely often. In particular t € R^jy-Si- 

(ii) Take a joint strategy s. Note that if all payoffs in s are > 0, then s is a Nash 
equilibrium. Suppose that some payoff in s is < 0. Then repeatedly select the 
first player in the cyclic order whose payoff is negative and let him switch to to- 
After at most n steps the Nash equilibrium (to, ■ ■ ., to) is reached. 

For the BR-improvement path we use the local scheduler / associated with 
the identity permutation, i.e., we repeatedly schedule the first player in the cyclic 
order who did not select a best response. 

Consider a joint strategy s taken from a BR-improvement path. Observe that 
for all fc if Sfc 7^ t and pk(s) > (so in particular if Sk is a best response to 
s-k), then Sk = S/tei- So for all i > 1, the following property holds: 

Z(i): if f(s) — i and Sj_i ^ to then for all j G {n, 1, . . .,i — 1}, Sj = Sj_i. 

In words: if i is the first player who did not select a best response and player 
i — 1 strategy is not to , then this is a strategy of every earlier player and of player 
n. 

Along each BR-improvement path that respects / the value of f(s) strictly 
increases until the path terminates or at certain stage f(s) = n. Note that then 
Sn-i = to since otherwise on the account of property Z(n) all players' strategies 
are equal, so s is a Nash equilibrium and hence f(s) is undefined. So the unique 
best response for player n is to. This switch begins a new round with player 1 as 
the next scheduled player. Player 1 also switches to t and from now on every 
consecutive player switches to t , as well. The resulting path terminates once 
player n — 2 switches to to. 

Consequently the length of the generated BR-improvement path is < 2n — 2. 

□ 
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It is easy to see that s is a Nash equilibrium in G coor d iff it is of the form 
(t, . . .,t). So we can alternatively formulate the first item as: the game 

G coord 

has the FIP iff n = 2 or it has exactly one Nash equilibrium. 
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